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Measures  of  Research 

By  W.  W.  CHARTERS 

TO  APPRAISE  exactly  the  value  of  educational  research  is 
an  insoluble  problem  at  the  present  time.  Its  nearest 
approximation  is  an  evaluation  of  those  aspects  of  a 
study  which  lend  themselves  to  quantitative  statement.  An 
educational  research  bureau,  such  as  that  established  at  Ohio 
State  University,  in  some  respects  is  situated  in  a  favorable  posi¬ 
tion  for  measurement,  because  half  or  more  of  its  effort  is 
devoted  to  service  projects  which  are  sometimes  easier  of  evalu¬ 
ation  than  more  fundamental  studies.  For  instance,  in  the 
Bureau  mentioned  more  than  half  of  the  funds  allotted  is  spent 
by  the  editorial  office,  the  reference  library,  and  the  appoint¬ 
ments  office.  One  can  measure  these  activities  more  easily  in  a 
superficial,  practical  sense,  at  least,  than  he  can  estimate  the 
values  of  other  activities.  The  service  functions  of  the  Bureau 
are  not  confined  to  these  three  departments  j  each  division  per¬ 
forms  a  considerable  number  of  services,  so  that  the  title  of 
the  Bureau  would  be  more  nearly  descriptive  of  its  activities 
if  it  were  called  a  bureau  of  research  and  service. 

We  must  acknowledge  that  there  is  absolutely  no  method 
of  measuring  the  effectiveness  of  the  selection  of  projects  to  be 
studied.  When  we  select  one  project  and  reject  another,  we 
cannot  say  that  we  were  either  wise  or  unwise,  because  we  do 
not  know  what  we  might  have  achieved  had  we  selected  the 
second.  Choice  is  frankly  based  upon  a  guess  which  is  steadied 
by  experience  and  knowledge. 

One  general  principle  of  selection,  which  the  Bureau  has 
followed  as  far  as  possible,  is  to  study  the  improvements  of 
techniques  and  instruments  rather  than  to  discover  new  knowl¬ 
edge,  in  so  far  as  such  a  principle  can  apply.  For  instance, 
the  Bureau  has  chosen  to  develop  the  Dale  Vocabulary 
List,  in  which  the  degrees  of  difficulty  of  10,000  words  for 
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children  of  Grades  IV,  VI,  and  VIII  are  established,  because 
it  is  a  tool  that  can  be  used  by  textbook  writers  and  hundreds 
of  thousands  of  teachers.  For  the  same  reason,  also,  it  has 
devoted  its  attention  to  the  development  of  new  techniques  of 
test  construction  in  which  actual  tests  are  by-products;  or  it  has 
spent  its  time  upon  the  production  of  new  techniques  of  bibli¬ 
ographical  reference  in  The  Personnel  Bibliografkical  Index 
rather  than  upon  the  study  of  a  few  specific  problems  in  the 
personnel  field.  This  principle  we  believe  to  be  wise,  but  we 
cannot  prove  its  wisdom. 

Similarly,  in  the  service  field  the  Bureau  has  deliberately 
selected  its  problems  in  those  areas  which  make  it  a  “trouble 
shooter.”  This  service  is  performed  for  the  Cleveland  Teachers 
Federation  and  the  Board  of  Education  where  the  future  of  the 
salary  schedule  is  at  stake.  The  Xenia  Orphans’  Home  with 
600  inmates  calls  in  the  Bureau  when  it  cannot  decide  upon  the 
proper  curriculums  for  its  students,  and  the  State  Department 
of  Education  asks  the  Bureau  to  set  up  school-building  pro¬ 
grams  for  state-aid  districts.  Projects  such  as  these  are  by  policy 
given  precedence  in  the  policy  of  the  Bureau  on  the  grounds 
that  the  state  needs  in  many  matters  of  major  importance  to 
the  educational  agencies  of  the  commonwealth  an  organization 
equipped  to  provide  technical  service. 

SUCH  policies  are  open  to  question,  and  no  one  can  prove  their 
wisdom;  but  once  the  projects  have  been  selected  there  are 
five  measures  which  can  be  applied,  although  all  cannot  be 
applied  to  each  project.  The  ultimate  standard  is  the  effect 
which  a  conclusion  reached  through  investigation  will  have 
upon  professional  practice.  To  apply  this  test  one  must  discover 
the  cases  in  which  practices  have  been  changed  as  a  result  of  a 
study;  also,  one  must  determine  the  importance  of  the  practices 
in  which  changes  have  been  wrought.  If  a  connection  between 
a  study  and  widespread  changes  in  significant  practices  can  be 
traced,  then  the  decision  that  the  study  was  of  value  can  be 
justified. 

Difficulties  in  applying  this  measure  are  numerous.  Teachers 
often  are  not  able  to  identify  the  source  of  change.  A  research 
discovery  may  be  used  by  a  teacher  who  was  told  it  by  a  super¬ 
visor  who  read  the  report  of  the  study;  or  he  may  copy  it  from 
another  teacher;  or  the  teacher  may  have  read  the  report  but 


OCTOBER  12,  1932 


3” 


56 

is 

15 

Df 

as 

i- 

?x 

16 
VQ 

ly 

il6 

:rs 

h6 

th 

h6 

;nt 

0- 

iq: 

ids 

to 

on 


6ir 

ir6 

b6 

6Ct 
IV6 
V6r 
if  a 
ic6S 
66n 
b6 
b6 

i6rs 

,rch 

)6r- 

•om 

but 


not  recall  that  he  has  read  it.  Only  in  spectacular  cases  are  we 
able  to  identify  the  influences  of  the  study  among  the  practices 
of  the  schoolroom  and  the  administrator’s  office.  It  is  possible, 
however,  to  discover  the  uses  to  which  people  say  they  put 
material.  For  instance,  school  principals  and  superintendents 
report  that  they  have  quoted  a  study  in  the  Educational  Re¬ 
search  Bulletin  at  teachers’  meetings.  Data  of  this  sort  are 
rough  measures  of  the  values  of  a  study,  but  there  is  no  guar¬ 
anty  that  schoolroom  practice  is  affected  though  we  hope  that 
good  will  flow  therefrom. 

A  second  measure,  which  should  be  applied  with  qualifi¬ 
cations  as  to  its  value  as  an  exact  instrument,  is  statistical:  tabu¬ 
lations  can  be  made  of  the  number  of  bulletins  distributed, 
positions  filled,  cards  catalogued,  communities  surveyed,  and  the 
like.  In  a  few  cases  it  is  possible  to  express  monetary  measures 
such  as  the  amounts  of  money  saved  for  students,  the  sizes  of 
bond  issues  carried,  and  other  financial  figures.  Statistical  meas¬ 
ures  are  easily  applied  if  records  are  kept,  but  obviously  they 
do  not  directly  measure  the  quality  of  the  services  measured. 
Such  figures  as  can  be  collected,  however,  are  of  use,  because 
the  services  frequently  are  assumed  to  be  valuable  by  all  or  by 
most,  and  in  such  cases  the  quantity  of  service  is  a  good  measure. 
If,  however,  the  value  of  the  service  is  not  admitted,  quantity 
will  not  carry  conviction. 

A  third  type  of  measure  which  is  relatively  easy  to  apply 
is  the  influence  of  a  study  upon  decisions  of  people.  When  an 
important  controversial  question  is  settled  by  a  study,  we  may 
assume  that  the  investigation  has  been  worth  while.  For  in¬ 
stance,  the  faculty  of  Ohio  State  University  was  divided  on  the 
relative  merits  of  the  quarter  and  the  semester  organization  of 
instruction.  The  Bureau  made  a  study  in  which  exhaustive  data 
were  collected.  When  as  a  result  the  faculty  overwhelmingly 
agreed  to  retain  the  quarter  system,  the  study  had  proved  its 
worth. 

To  these  measures  may  be  added  a  fourth.  It  is  possible  to 
collect  examples  of  references  to  studies  in  technical  literature. 
For  instance,  Buckingham’s  study  of  supply  and  demand  seems 
from  a  canvass  of  the  literature  in  this  area  to  have  set  the  pat¬ 
tern  for  a  score  of  later  studies.  This  measure — references  in 
the  literature — is,  however,  exceedingly  difficult  to  administer, 
because  of  the  huge  task  of  reading  the  literature  and  tabulating 
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the  items;  so  that  we  may  safely  assume  that  the  references  to 
a  study  are  several  times  greater  than  the  instances  collected. 
Except  in  a  few  outstanding  studies  a  canvass  of  the  literature 
would  scarcely  pay  for  the  effort  involved  in  collection. 

Finally,  one  may  collect  the  opinions  of  users.  Evaluations 
of  the  services  may  be  volunteered  or  asked  for  and  recorded. 
When  a  college  is  helped  in  reorganizing  its  curriculum,  the 
recorded  opinion  of  the  dean  and  the  faculty  has  some  validity. 
The  library  service  may  be  appraised  by  faculty  and  students, 
by  investigators  who  travel  distances  to  consult  the  catalogue, 
and  by  reputable  institutions  who  spend  large  sums  of  money  to 
photograph  the  catalogue.  Opinions  of  value  are  easy  to  secure, 
and  if  inquiries  are  not  biased  the  returns  possess  substantial 
validity. 

These  five  measures  are  applied  in  the  evaluations  of  the 
Bureau  activities  from  time  to  time  and  a  system  has  been 
inaugurated  in  which  an  attempt  is  made  to  evaluate  each  study 
as  soon  as  its  value  or  uselessness  has  begun  to  become  apparent. 
With  due  attention  to  the  use  of  all  measures,  however,  chief 
reliance  in  support  of  the  organization  must  be  placed  upon  an 
attitude  favorable  to  research  and  service  of  the  type  which  the 
Bureau  conducts.  This  attitude  is  influenced  by  such  data  as 
the  five  measures  provide. 

SEVERAL  measures  of  evaluation  have  been  applied  to  the 
work  of  the  Reference  Division.  Its  chief  function  since 
the  organization  of  the  Bureau  of  Educational  Research  in  1921 
has  been  to  build  up  the  library  of  the  Bureau  of  Educational 
Research.  The  staff  regularly  consists  of  the  librarian  in  charge 
of  the  Division,  a  cataloguer,  the  reference  assistant,  and  a 
stenographer,  with  occasional  part-time  assistants.  When  Mr. 
Buckingham  organized  the  Bureau,  his  plans  included  the 
development  of  a  comprehensive  bibliographical  service  for  the 
use  of  investigators.  In  pursuance  of  this  plan,  persistent  effort 
has  been  directed  toward  the  collecting  and  cataloguing  of  the 
contents  of  journals  and  of  technical  reports  wherever  obtain¬ 
able.  Less  attention  has  been  paid  to  the  gathering  of  books, 
because  funds  have  been  scarce  and,  also,  because  the  library  of 
the  College  of  Education  chiefly  has  been  concerned  with  the 
acquisition  of  books.  Indeed,  the  amount  of  money  spent  for 
such  purposes  has  been  small;  the  materials  have  been  secured 
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0  almost  entirely  through  the  good  services  of  the  Educational 

1.  Research  Bulletin,  the  Journal  of  Educational  Research, 

e  and  the  Journal  of  Higher  Education  through  exchanges  and 
the  miscellaneous  pamphlets  which  such  journals  attract  and 
is  through  the  annual  textbook  exhibit. 

i.  By  applying  statistical  evaluation  we  have  secured  the  fol- 

le  lowing  data.  There  are  230,000  cards  in  the  subject  index. 

y.  Additions  of  about  25,000  are  being  made  annually.  Originally 

s,  the  contents  of  300  educational  magazines  were  catalogued;  but 

e,  since  the  Educational  Index  has  been  published — since  1929 — 

to  the  cataloguing  load  has  been  lightened  by  its  use,  although 

e,  with  this  aid,  more  cards  have  been  added  to  the  files  than 

al  before  its  establishment.  This  is  due  to  the  increase  in  the 

volume  of  the  literature. 

be  The  library  is  used  by  about  21,000  persons  during  a  year; 

sn  that  is  to  say,  2 1 ,000  slips  are  filled  out,  but  probably  half  the 

ly  use  of  the  library  is  not  recorded  on  slips.  Fifty  per  cent  of  the 

at.  slips  are  filled  out  during  the  regular  session  by  faculty  mem- 

ief  I  bers  of  the  Bureau  and  the  College  of  Education.  Summer- 

an  j  session  students  pack  the  inadequate  reading  room  beyond  the 

he  I  point  of  effective  use.  Approximately  50  bibliographies  a  year 

as  I  are  provided  for  correspondents,  averaging  four  typewritten 

I  pages  in  length.  Five  bibliographical  monographs  have  been 

I  published  since  the  installing  of  the  library  in  1921.  The  com- 

:he  puted  value  of  the  library  is  $21,000,  but  this  is  not  an  adequate 

ice  representation  of  its  value,  because  much  of  this  material  has 

i2i  been  received  without  charge,  and  much  of  it  cannot  be  replaced, 

aal  The  librarian  carries  a  small  teaching-load  in  summer  classes 

[■ge  for  school  librarians. 

I  a  Other  measures  of  evaluation  have  been  applied  to  the  Ref- 

dr.  I  erence  Division.  When  the  Educational  Index  was  established 
the  in  1929  by  the  H.  W.  Wilson  Company,  the  subject-index 

the  headings  used  in  the  cataloguing  of  the  library  were  adopted, 

ort  and  Miss  Voegelein  was  drafted  to  install  the  Index.  Investi- 

the  gators  from  other  institutions  have  visited  the  library  from  time 

lin-  to  time  attempting  to  trace  references  in  connection  with  studies 

)ks,  which  they  have  thought  important.  Teachers  College  of 

f  of  Columbia  University  has  photographed  the  cards  from  the 

the  subject  index  to  add  to  their  subject  catalogue, 
for  As  another  means  of  evaluation  an  inquiry  was  made  of  all 

ired  the  faculty  members  of  the  College  of  Education  asking  their 
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judgment  of  the  value  of  the  library  to  them  and  asking,  also, 
for  suggestions  toward  improvement.  Some  suggestions  were 
made,  but  on  the  whole  the  members  of  the  staff  of  the  College 
of  Education  were  most  enthusiastic.  Closer  co-ordination  with 
the  library  of  the  College  of  Education  is  the  pressing  need. 

The  library  is  a  service  agency — a  tool  for  research  workers. 
A  few  bibliographies  have  been  published,  but  the  chief  research 
contribution  of  the  Reference  Division  is  the  Subject  Heading 
List  prepared  by  Miss  L.  Belle  Voegelein. 

The  Appointments  Division  has  become  the  largest  teach¬ 
ers’  agency  in  Ohio.  Each  year  600  heads  of  schools  seek 
aid  in  selecting  teachers  to  fill  1,500  vacancies  in  their  staffs. 
Two-thirds  of  these  requests  are  from  schools  and  colleges  of 
the  state;  they  have  come  from  every  county,  from  99  of  the 
109  cities,  39  of  the  49  exempted  villages,  40  of  65  colleges  of 
Ohio,  and  from  38  other  states  238  school  districts  have  sought 
recommendations.  In  42  other  states  requests  for  instructors 
have  come  from  302  colleges,  for  the  placing  of  college  teachers 
is  an  increasing  activitity  of  the  Division. 

Measured  financially  the  Division  at  commercial-agency 
rates  is  more  than  self-supporting  although  the  service  is  pro¬ 
vided  for  the  schools  of  the  state  without  cost  to  the  teachers. 
The  amount  of  work  involved  in  the  service  each  year  is  sub¬ 
stantial — 2,000  interviews,  2,500  letters  written,  4,000  cre¬ 
dentials  sent  to  officers,  and  12,000  pieces  of  first-class  mail 
received.  The  quality  of  the  service  was  appraised  by  written 
requests  to  293  school  administrators.  Of  42  per  cent  replying 
all  rated  the  service  as  “very  valuable”  or  “valuable.”  The 
service  was  judged  “prompt”  by  all,  the  information  “ade¬ 
quate”  by  practically  all,  and  the  recommendations  “depend¬ 
able”  by  a  large  majority.  The  general  opinion,  however,  that 
the  service  is  good  is  qualified  by  the  fact  that  58  per  cent  of 
those  addressed  did  not  reply.  This  is  an  uncontrolled  factor  in 
the  appraisal.  The  Division  is  unable  to  place  all  those  who 
want  positions.  This  is  partly  due  to  the  national  oversupply  of 
high-school  teachers. 

The  Appointments  Division  has  been  productive  in  research. 
Two  monographs,  three  pamphlets,  and  fifty-two  articles  have 
appeared  during  the  last  four  years.  Most  significant  is  the 
group  of  teacher  supply-and-demand  studies.  Mr.  Buckingham, 
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while  in  charge  of  the  Bureau,  began  the  studies  in  this  field, 
which  he  reported  in  the  monograph.  Supply  and  Demand  in 
Teacher  Training.  This  has  set  a  pattern  followed  by  many 
investigators  in  other  states.  Mr.  Anderson,  the  present  head 
of  the  Division,  has  continued  the  study  of  demand  and  supply 
in  Ohio  partly  to  guide  students  and  partly  to  inform  univer¬ 
sities  and  teachers’  colleges,  but  chiefly  to  perfect  a  formula  to 
forecast  demand.  Should  this  prove  feasible  it  will  be  possible 
to  predict  for  Freshmen  how  many  teachers  will  be  needed  to 
supply  the  demand  that  will  exist  in  a  particular  subject-matter 
field  when  they  graduate. 

Turning  to  the  editorial  functions  of  the  Bureau  we  find 
that  during  its  ten  years  of  service  it  has  edited  continu¬ 
ously  the  Educational  Research  Bulletin,  the  Journal  of 
Educational  Research  until  1928,  and  the  Journal  of  Higher 
Education  since  its  beginning  in  1930.  In  addition  to  these 
journals  it  has  edited  16  monographs  j  9  volumes  of  the  Pro¬ 
ceedings  of  the  Ohio  State  Educational  Conference;  3  issues 
of  Education  on  the  Airy  the  yearbook  of  the  Institute  for 
Education  by  Radio;  and  cared  for  many  miscellaneous 
publications. 

Any  evaluation  of  each  publication  included  in  “Bureau 
of  Educational  Research  Monographs”  is  extremely  difficult 
to  secure  beyond  figures  of  distribution.  The  numbers  sold 
are  a  poor  criterion  because,  as  university  publications,  many 
copies  are  exchanged  with  other  institutions,  and  many  are 
given  away  as  matters  of  policy;  these  depreciated  figures 
are  offset  by  the  fact  that  the  copies  in  libraries — libraries  are 
the  best  purchasers  of  the  monographs — are  used  by  many 
readers.  The  monograph  series  is  not  nearly  self-supporting. 
In  a  few  cases  some  evidences  of  significant  use  are  discoverable, 
as  in  the  case  of  the  monographs  by  Miss  Voegelein  and  Mr. 
Buckingham  already  referred  to.  No  attempt  is  made  to  sell 
the  Proceedings  of  the  Educational  Conference,  but  the  first 
two  issues  of  Education  on  the  Air  have  been  self-supporting. 

The  Journal  of  Educational  Research  ceased  to  be  edited  in 
the  Bureau  when  Mr.  Buckingham  resigned,  and  no  attempt 
has  been  made  to  measure  its  value.  The  Journal  of  Higher 
Education  is  a  young  magazine  which  is  underwritten  by  Ohio 
State  University  and  is  not  yet  completely  self-supporting.  A 
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few  measures  were  assembled.  During  the  last  year  and  a  half 
the  list  of  paid  subscribers  has  been  doubled  in  spite  of  the 
depression.  All  the  states  of  the  Union  and  several  foreign 
countries  are  represented.  One-third  of  its  subscribers  are 
libraries}  14  per  cent  of  its  subscribers  are  professors}  12  per 
cent  are  administrative  officers,  other  than  deans  and  presidents} 
10  per  cent  are  presidents}  and  9  per  cent  are  deans.  Its  con¬ 
tributors  are  connected  with  more  than  74  institutions  from  31 
states  and  one  foreign  country.  In  a  recent  bibliography  of 
experimental  studies  made  by  individuals  in  higher  institutions 
32  of  the  59  citations  were  references  to  articles  which  had 
appeared  in  the  Journal  during  its  two  years  of  publication. 
Opinions  of  the  quality  of  the  Journal  were  not  sought,  but 
many  unsolicited  letters  of  favorable  tenor  have  been  received. 

A  more  sustained  attempt  has  been  made  to  measure  the 
worth  of  the  Educational  Research  Bulletin.  This  bul¬ 
letin  is  issued  about  18  times  a  year  and  is  distributed  without 
cost  to  those  who  wish  to  receive  it.  No  annual  attempt  has 
been  made  to  secure  a  general  measure  of  its  value,  but  many 
unsolicited  letters  of  commendation  have  been  received  each 
year.  More  than  I5,(X)0  copies  of  the  Bulletin  are  sent  out 
annually  in  response  to  special  requests.  To  keep  the  mailing 
list  within  the  appropriation  for  the  Bulletin  the  names  of 
persons  who  do  not  write  in  to  say  that  they  wish  to  have  their 
names  continued  on  the  list  are  dropped  each  fall  from  the 
mailing  list,  which  at  present  consists  of  5,700  names  and 
addresses. 

To  every  third  person  on  the  list  an  inquiry  was  directed 
last  May.  Of  this  one-third,  30  per  cent,  replied  from  44 
states.  We  therefore  received  data  from  one-tenth  of  our  sub¬ 
scribers.  Assuming  that  the  sampling  was  random,  the  judg¬ 
ment  of  this  30  per  cent  of  one-third  would  be  typical  of  30  per 
cent  of  all  receivers  of  the  Bulletin.  What  the  opinions  of  the 
other  70  per  cent  are  is  of  course  conjectural.  They  probably 
are  on  the  whole  not  so  favorable  as  those  who  replied,  since 
readers  do  not  so  freely  write  critical  as  commendatory  state¬ 
ments.  College  teachers  of  education  sent  the  most  letters, 
while  superintendents,  principals,  and  other  administrators 
ranked  second.  There  were  fewest  replies  from  classroom 
teachers,  because  there  is  a  smaller  proportion  of  teachers’ 
names  on  the  Bulletin  mailing  list  than  of  the  other  groups. 
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But  since  most  of  the  superintendents  and  principals  who  replied 
said  that  they  put  the  Bulletin  in  the  teachers’  library,  circu¬ 
lated  marked  copies,  and  used  it  as  the  basis  of  discussion  for 
teachers’  meetings,  it  is  likely  that  a  much  larger  number  of 
teachers  are  familiar  with  the  Bulletin  than  the  proportion  of 
classroom  teachers’  names  on  the  mailing  list  would  indicate. 

Of  the  577  persons  replying,  all  the  correspondents  stated 
that  the  Bulletin  was  a  contribution  to  education  except  five 
principals,  two  board  members,  one  superintendent,  and  one 
professor  not  in  a  college  of  education.  The  Editors  can  con¬ 
servatively  assume,  therefore,  that  at  least  30  per  cent  of  its 
6,000  readers,  or  1,800,  believe  the  Bulletin  to  be  of  sub¬ 
stantial  value.  This  number  is  larger  than  the  list  of  subscribers 
of  most  journals  of  its  type. 

The  uses  to  which  the  issues  are  put  were  canvassed  by 
asking  statements  from  the  correspondents;  no  statements  were 
suggested  on  the  simple  questionnaire.  The  Bulletin  was  used 
for  practical  purposes,  such  as  changes  in  practice,  suggestions 
for  research,  and  to  keep  informed;  for  discussions  in  teachers’ 
meetings;  for  material  for  speeches;  references  in  bibliog¬ 
raphies;  material  for  class  use;  and  as  a  source  for  new  ideas. 

For  several  years  the  Bureau  has  provided  a  survey  service. 

When  a  school  district  wishes  to  develop  a  long-term  build¬ 
ing  program,  members  of  the  Survey  Division  of  the  Bureau, 
upon  request,  will  study  its  buildings  and  indicate  where  new 
buildings  should  be  placed  in  terms  of  population  trends  and 
what  size  and  type  of  building  should  be  built,  as  well  as 
changes  to  be  made  in  the  buildings  already  in  use.  This  service 
has  been  used  by  the  State  Department  of  Education  in  its 
rehabilitation  program;  by  many  middle-sized  cities  such  as 
Marion  and  Springfield,  and  by  numerous  smaller  districts  in 
the  state.  The  Division  also  collects  each  year  certain  items  of 
financial  information  about  special  levies  and  school  bond  issues 
and  after  tabulating  them  distributes  them  to  school  officials. 
In  all,  567  communities  in  83  of  the  88  counties  of  the  state 
have  received  services  of  some  kind  from  this  Division. 

Twenty-nine  surveys  of  school  districts  with  a  combined 
population  of  320,000  people  have  been  made,  these  have 
involved  $7,250,000  which  were  proposed  to  the  voters  as  bond 
issues,  and  91  per  cent  were  passed.  Architects’  plans  for 
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$5,000,000  worth  of  buildings  have  been  checked.  For  the 
State  Department  of  Education  comprehensive  studies  were 
made  to  determine  the  location  and  type  of  buildings  needed 
in  its  two-year  rehabilitation  program,  and  some  $8,000,000 
worth  of  buildings  were  recommended  and  checked.  On  the 
basis  of  the  usual  architects’  fees  of  one-half  of  one  per  cent, 
the  Bureau  “earned”  about  $65,000,  which  would  have  much 
more  than  paid  for  the  total  cost  of  this  Division  of  the  Bureau 
had  any  fees  been  charged. 

Certain  major  surveys  of  a  more  general  type  have  been 
made  by  the  Bureau.  At  the  request  of  the  Board  of  Trustees 
of  the  University  a  survey  of  instruction  was  conducted  by  Mr. 
Buckingham  in  1927.  Just  what  changes  resulted  from  this 
survey  is  difficult  to  determine  beyond  the  installation  of  a 
system  of  instructional  accounting  and  analysis.  He  also,  in 
1928,  at  the  request  of  the  state  director  of  education  and  the 
Board  of  Trustees  undertook  a  survey  of  Kent  Teachers  Col¬ 
lege.  The  college  administration  was  completely  reorganized, 
and  some  twenty-five  specific  recommendations  have  been 
adopted.  In  1932  the  Bureau  conducted  a  study  of  teachers’ 
salaries  at  Cleveland  for  the  Cleveland  Teachers  Federation  in 
co-operation  with  the  Board  of  Education.  As  a  result  of  this 
study  recommendations  were  made  which  proved  acceptable  to 
both  the  teachers  and  the  Board  of  Education. 

During  the  year  letters  were  addressed  to  the  recipients  of 
these  services  asking  them  about  the  value  of  the  services  ren¬ 
dered.  More  than  seven  hundred  replies  were  received  from 
all  sources,  which  were  almost  unanimously  favorable. 

IN  THE  curriculum  field  a  number  of  studies  have  been  under¬ 
taken.  The  director  of  the  Bureau  is  chairman  of  the  Cur¬ 
riculum  Committee  of  the  College  of  Education,  which  for 
three  years  has  been  working  upon  the  revision  of  the  cur¬ 
riculum  for  the  training  of  high-school  teachers.  Numerous 
studies  have  been  made  by  committees  of  the  faculty,  and  certain 
changes  have  resulted}  but  since  the  work  is  not  yet  completed, 
no  evaluation  is  possible.  The  College  of  Veterinary  Medicine 
is  being  assisted  in  the  reorganization  of  its  curriculum. 
Numerous  changes  have  been  made,  and  condensations  to  the 
extent  of  10  per  cent  of  the  number  of  hours  necessary  to  cover 
the  course  of  training  for  veterinarians  are  already  in  effect. 


OCTOBER  12,  1932  319 

An  important  curriculum  instrument  has  been  developed 
— a  vocabulary  difficulty  list  which  indicates  the  percentage  of 
children  of  Grades  IV,  VI,  and  VIII  who  know  the  meanings 
of  the  10,000  most  common  words.  This  has  been  used  for 
experimental  purposes,  but  it  has  not  been  subjected  to  the  test 
of  wide  use. 

The  Division  of  Tests  and  Measurements  is  developing  new 
type  tests  to  measure  objectives  of  teaching  other  than  the 
mastery  of  information.  Work  is  being  carried  on  with  fourteen 
departments  on  the  campus  including  botany,  zoology,  and 
home  economics.  The  results  are  promising.  Funds  have  been 
provided  to  supplement  university  appropriations  by  one  of  the 
foundations,  and  co-operation  on  a  national  scale  has  been 
received.  But  not  enough  time  has  elapsed  to  permit  of  evalu¬ 
ation  other  than  the  keen  interest  of  experts  in  testing. 

IN  THE  Personnel  Division  a  new  type  of  bibliographical  instru¬ 
ment  has  been  developed  which  is  incorporated  in  the  re¬ 
cent  Bureau  publication,  T he  Personnel  Bibliogra'phical  Index. 
Many  letters  of  commendation  have  been  received,  which  have 
been  unusually  enthusiastic.  In  the  personnel  field  the  Division 
has  worked  out  techniques  for  manualizing  routines,  by  which 
all  the  activities  of  an  office  are  reduced  to  writing  by  the 
workers  and  put  into  manual  form.  This  procedure  saves  an 
enormous  amount  of  time  and  results  in  a  substantial  elimina¬ 
tion  of  errors.  The  judgment  of  all  organizations  assisted  in 
this  way  is  unanimously  favorable.  The  techniques  are  so  simple 
that  one  wonders  why  such  job  analyses  have  not  been  made 
by  all  organizations  which  use  systematic  routines. 

The  Radio  Division  has  been  established  for  only  one  year. 

In  1931  the  Payne  Fund  provided  the  necessary  money 
for  five  years  of  experimental  study.  Except  for  the  annual 
Institute  for  Education  by  Radio  it  is  not  possible  to  evaluate 
its  activities.  The  Institute,  in  the  opinion  of  the  persons  who 
participated,  was  a  success. 

SINCE  1921,  when  the  Bureau  was  established,  340  service 
and  research  studies  have  been  announced  from  time  to 
time.  Of  these,  72  were  not  worked  upon  or  were  discontinued, 
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170  were  completed  to  the  point  of  publication  or  report,  and 
98  are  still  in  progress.  The  projects  are  classified  roughly 
along  the  lines  of  the  divisions  of  the  Bureau  organization.  The 
number  of  discontinued  projects  is  theoretically  too  large  j  there 
should  be  none.  They  were  caused  by  changes  of  personnel  in 
the  staff,  too  great  ambition  in  the  autumn,  and  changing  con¬ 
ditions  of  work  in  the  Bureau. 

For  four  years,  with  funds  provided  by  the  Payne  Fund, 
the  Bureau  has  been  co-operating  in  a  series  of  studies  on  the 
influence  of  moving  pictures  upon  children.  These  studies  are 
just  now  being  completed,  and  their  evaluation  will  necessarily 
be  delayed  until  a  few  years  have  elapsed. 

IN  CONCLUSION,  the  members  of  the  Bureau  staff  have  used  all 
available  varieties  of  measures  upon  these  selected  groups 
of  activities.  Statistics  are  the  easiest  measure  to  use.  The 
quality  of  the  product  can  be  measured  in  terms  of  the  opinions 
of  the  consumers.  The  uses  to  which  studies  are  put  can  be 
easily  determined  in  administration,  but  not  so  easily  in  other 
fields.  The  influence  of  the  studies  on  practice  in  general  is 
extremely  difficult  to  determine.  The  staff  of  the  Bureau  with 
the  data  from  this  survey  before  them  feel  that  in  the  mass  the 
measure  of  value  is  substantial. 


Reference  Division 

By  RUTH  SEEGER 

SINCE  September  i,  1931,  the  Reference  Division  has  func¬ 
tioned  almost  entirely  as  a  service  division  of  the  Bureau. 
With  Miss  Lingenfelter’s  resignation  taking  effect  Octo¬ 
ber  I  we  have  had  a  staff  shortage  which  brought  to  a  standstill 
such  projects  as  had  been  planned  for  the  year.  We  have  to 
the  credit  of  the  Bureau  staff  only  one  publication,  “Bibliog¬ 
raphy  of  Fiction  Pertaining  to  Vocations,”  which  was  compiled 
and  annotated  by  Mary  R.  Lingenfelter  and  Alice  Hanson.‘ 
Services  to  the  faculty  and  student  body  have  shown  con¬ 
stant  growth.  Attendance  statistics  for  the  past  year  revealed 
that  during  the  nine-month  school  year  our  average  weekly 

^  Mimeographed  by  the  American  Library  Association  in  the  fall  of  1931. 
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attendance  was  298,  and  that  the  entire  attendance  for  the 
twelve  weeks  of  the  Summer  Quarter  was  10,704,  which  is  a 
weekly  average  of  895 — a  growth  in  library  usage  over  previous 
years. 

The  public  at  large  is  served  chiefly  by  the  supplying  of 
bibliographies  or  information  in  response  to  written  requests. 
During  the  last  twelve  months  47  requests  were  answered  by 
supplying  bibliographies  j  and  23  by  supplying  information. 

No  library  report  would  be  complete  without  reference  to 
the  growth  of  its  collection.  We  find  that  during  the  past 
year  the  library  has  received  the  following  publications: 


Books .  1)038 

Pamphlets .  539 

Instructional  aids .  390 

Courses  of  study .  296 

College  catalogues .  204 

Reports .  63 

Directories .  60 

Tests .  43 

Miscellaneous  materials .  203 

Total .  2)836 


These  accessions  are  in  addition  to  our  periodicals  which  are 
received  regularly.  At  the  present  time  this  periodical  list  num¬ 
bers  310. 

One  of  the  unique  features  of  our  service  is  the  index¬ 
ing  of  all  such  material  received  as  is  not  already  indexed  in 
the  Education  Index  published  by  the  H.  W.  Wilson  Company. 
Since  they  index  only  176  periodicals  this  leaves  144  to  be 
indexed  here.  In  addition,  many  pamphlets  and  reports  are 
indexed,  greatly  increasing  the  volume  of  such  work.  Entries 
to  our  index  file  during  the  past  year  have  brought  the  total 
number  to  approximately  230,000.  Early  in  1932,  Teachers 
College,  Columbia  University,  approached  us  on  the  possibility 
of  copying  this  subject  index.  Arrangements  were  made  whereby 
such  an  undertaking  was  made  possible. 

There  has  been  a  growing  conviction  on  the  part  of  our 
staff  that  closer  co-operation  with  the  Education  Library  would 
be  advantageous  in  both  improved  service  and  saving  of  time. 
With  this  point  in  view,  we  have  undertaken  the  recataloguing 
of  our  entire  collection  of  professional  books.  The  Library  of 
Congress  classification  has  been  adopted  to  conform  to  the 
usage  of  the  other  libraries  on  the  campus.  Their  subject- 
headings,  with  slight  modifications,  have  also  been  adopted. 


322 


EDUCATIONAL  RESEARCH  BULLETIN 


Appointments  Division 

By  EARL  W.  ANDERSON 

The  Appointments  Office  is  the  connecting  link  between 
those  who  train  and  those  who  employ  teachers.  Its 
major  function  is  to  locate  most  advantageously  to  the 
schools  and  to  the  applicants  themselves  those  who  are  trained 
by  Ohio  State  University. 

During  1931,  the  Appointments  Division  has  directly  aided 
210  teachers  in  securing  positions}  105  of  these  were  placed 
directly,  and  105  were  assisted  since  detailed  data  regarding 
them  were  furnished  to  employers.  The  office  received  916 
!  calls  for  teachers,  recommended  1,945  persons,  sent  out  4,131 

sets  of  credentials,  held  2,304  interviews,  received  10,445 
pieces  of  first-class  mail,  wrote  2,028  letters,  and  typed  7,783 
pages  of  material. 

Almost  all  of  the  vacancies  during  1932  have  been  in  the 
I  small  school  systems.  The  larger  public  schools  and  the  col¬ 

leges  for  the  most  part  have  reduced  their  personnel.  The 
continued  return  of  former  teachers  to  the  profession  and  the 
entrance  into  teaching  of  persons  who  in  normal  times  were  not 
interested  in  this  occupation,  together  with  the  actual  reduction 
in  the  number  of  teaching  positions,  have  increased  the  over¬ 
supply  of  trained  teachers  when  compared  with  the  figures  of 
1931.  Hence,  the  outlook  for  1932  placement  is  approximately 
60  per  cent  as  good  as  that  of  1931. 

By  the  end  of  1932  there  will  be  on  file,  available  to  school 
administrators,  detailed  records  of  the  training,  experience,  and 
qualifications  of  more  than  3,300  teachers  and  prospective 
teachers  trained  in  virtually  all  fields  of  instruction.  This  file 
is  being  used  more  and  more  by  school  administrators  to  secure 
information  regarding  teachers  who  have  been  recommended 
to  them  by  the  Appointments  Division  or  of  whom  they  have 
heard  from  other  sources. 

Twenty-four  published  reports  written  by  members  of  the 
Appointments  Division  between  September,  1931,  and 
September,  1932,  include  two  monographs  and  22  articles. 
One  monograph  containing  an  analysis  of  the  supply-demand 
situation  in  Ohio  for  the  year  1929-30  was  prepared  by  Mr. 
Anderson  and  R.  R.  Foster.  The  comparison  of  the  number 
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of  prospective  teachers  prepared  in  1929  with  the  number  of 
positions  available  during  the  subsequent  year  was  made  in 
co-operation  with  the  Ohio  Education  Association  and  the  State 
Department  of  Education.  Three  studies  by  O.  E.  Sibert  and 
Mr.  Anderson  reported  analyses  of  the  experience  and  training 
of  school  administrators  in  Ohio,  the  relation  of  their  experience 
and  training  to  salaries,  and  the  trends  in  the  number  of  women 
in  administrative  positions. 

Surveys  of  high-school  teaching  positions  in  23  states  fur¬ 
nished  data  as  to  how  widely  each  subject  is  taught  in  high 
school  and  to  what  extent  it  is  taught  alone  and  in  combination 
with  other  subjects.  From  this  information  reports  were  made 
by  Mr.  Anderson  and  R.  H.  Eliassen  covering  high-school 
teaching  positions  in  mathematics,  music,  foreign  languages, 
English,  industrial  arts,  and  home  economics.  The  same  authors 
reviewed  studies  made  in  teacher  supply  and  demand  during 
the  year  1931.  Mr.  Anderson  wrote  four  articles  on  various 
phases  of  the  supply-demand  situation  and  Mr.  Eliassen  con¬ 
tributed  several  articles  based  on  his  follow-up  study  of  the 
graduates  of  the  College  of  Education  over  a  five-year  period. 

Information  regarding  the  output  of  teacher-training  insti¬ 
tutions  for  the  years  1929-30  and  1930-31  has  been  collected, 
coded,  and  tabulated,  as  have  data  regarding  the  number  of 
new  teachers  in  the  state  for  the  years  1930-31  and  1931-32. 
During  the  next  year  the  studies  now  under  way  should  be 
completed  and  information  collected  for  the  year  1931-32.  It 
will  then  be  possible  to  have  facts  covering  a  three-year  period. 
This  will  permit  the  showing  of  some  trends  which  hitherto 
have  not  been  revealed. 

Editorial  Division 

By  JOSEPHINE  H.  MACLATCHY 

The  chief  function  of  the  Editorial  Division  is  the  editing 
of  the  regular  and  occasional  publications  of  the  Bureau, 
the  proofreading,  and  the  supervision  of  such  materials 
until  they  are  ready  for  distribution.  During  the  past  year  the 
Division  has  cared  for  1 8  issues  of  the  Educational  Research 
Bulletin  j  9  issues  of  the  Journal  of  Higher  Education;  the 
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Proceedings  of  the  Twelfth  Ohio  State  Educational  Conference 
(462  pages)  j  Education  on  the  Air,  1932  (Proceedings  of  the  | 
Third  Institute  for  Education  by  Radio)  j  T he  Personnel  Bib¬ 
liographical  Index  (433  pages)  j  and  four  monographs  belong¬ 
ing  to  the  Bureau  of  Educational  Research  Monograph  series, 
Teacher  Supply  and  Demand  in  Ohio  (170  pages),  Financial 
Ability  of  Ohio  City  and  Exempted-Village  School  Districts 
(74  pages),  Service  Studies  in  Higher  Education  (284  pages), 
and  Cleveland  Teachers*  Salaries  (196  pages). 

The  Editorial  Division  also  cares  for  the  sales  of  the  publi¬ 
cations  of  the  Bulletin  and  for  the  advertising  materials  for  the 
monograph  series,  for  the  Journal  of  Higher  Education y  and 
for  Education  on  the  Air.  The  books  sent  in  for  review  make 
another  responsibility  of  the  Editorial  Division.  During  the 
cations  of  the  Bureau  and  for  the  advertising  materials  for  the 
review  to  the  Educational  Research  Bulletin  by  76  pub¬ 
lishers,  and  79  books  by  38  publishers  have  been  sent  to  the 
Journal  of  Higher  Education. 

UNDER  the  direction  of  the  writer.  Miss  Elva  Lyon  has  been 
working  upon  the  problem  of  devising  standards  for  judg¬ 
ing  reviews  of  educational  books.  Miss  Lyon  has  studied  many 
book  reviews  in  the  fields  of  literature  and  science}  she  has 
analyzed  reviews  by  famous  critics  of  the  past  and  of  the 
present}  and  she  has  compared  reviews  written  by  different 
reviewers  of  the  same  book.  This  variety  of  effort  was  preceded 
by  a  careful  study  of  standard  works  on  book-reviewing. 

During  the  past  year  a  detailed  statistical  analysis  of  the 
records  obtained  from  1,242  six-year-olds  at  Cincinnati  on  the 
Number  Test  for  Pupils  Entering  Grade  One  has  been  con¬ 
tinued.  A  careful  study  has  been  made  of  the  records  of  these 
children  on  the  20  addition  combinations  given  in  the  test. 
Since  these  combinations  were  selected  from  a  table  showing  the 
difficulty  of  addition  combinations  in  ascending  order,  facts 
regarding  the  six-year-old’s  familiarity  with  these  combinations 
will  throw  light  on  the  difficulty  of  addition  combinations. 
Some  information  also  will  be  gained  regarding  the  order  in 
which  such  number  facts  are  acquired  in  the  unguided  experi¬ 
ence  of  the  pre-school  child.  This  material  with  the  other  facts 
already  gathered  from  these  data  will  soon  be  published  as  a 
“Bureau  of  Educational  Research  Monograph.” 
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A  nation-wide  study  of  the  equipment  and  administrative 
features  essential  to  the  articulation  of  kindergarten  and  pri¬ 
mary  practices  has  been  attempted.  The  response  from  the 
superintendents  and  supervisors  in  the  310  cities  in  which  some 
form  of  articulation  is  attempted  has  been  excellent.  A  report 
of  this  study  will  be  ready  for  publication  soon. 


Survey  Division 

By  T.  C.  HOLY 

The  work  of  this  Division  has  to  do  largely  with  the 
public  schools  of  the  state  and  consists  chiefly  of  various 
kinds  of  surveys  and  studies  and  the  collection  and 
distribution  of  financial  data  relating  to  the  public  schools. 

During  the  Winter  Quarter  the  writer,  in  co-operation  with 
H.  H.  Davis  and  Earl  W.  Anderson,  both  of  Ohio  State  Uni¬ 
versity,  made  a  study  of  teachers’  salaries  in  the  city  of  Cleve¬ 
land.  This  study,  which  has  now  been  published  as  Bureau  of 
Educational  Research  Monograph  No.  16,  was  built  around 
two  major  questions:  What  salaries  do  Cleveland  teachers 
merit?  What  salaries  can  the  Cleveland  school  district  afford 
to  pay?  In  addition  to  the  data  collected  around  these,  the 
probable  cost  of  the  present  salary  schedule  was  calculated,  and 
a  section  dealing  with  other  factors  bearing  on  the  salary  situa¬ 
tion  was  included. 


Studies  of  salaries  were  made  at  Cleveland  Heights  and  at 
Lakewood,  neither  of  which  will  be  printed.  W.  W.  Charters 
and  H.  A.  Toops  served  as  consultants  in  all  three  of  these 
studies. 

The  past  summer  the  Board  of  Trustees  of  the  Ohio  Sol¬ 
diers’  and  Sailors’  Orphans’  Home  at  Xenia,  Ohio,  passed  a 
resolution  providing  for  “a  general  survey  of  our  school  system 
to  assist  this  Board  in  making  plans  for  the  future  education  of 
these  children.”  After  conferences  with  the  Board  and  the 
superintendent  of  the  Home,  arrangements  were  made  whereby 
the  Bureau  of  Educational  Research  would  undertake  the  study. 
The  original  outline  of  this  survey  set  up  three  major  prob¬ 
lems:  an  appraisal  of  the  native  ability,  aptitudes,  and  interests 
of  the  pupils  in  the  school,  together  with  their  achievement  in 
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school  subjects 5  an  analysis  of  the  opportunities  and  the  occu¬ 
pations  for  which  the  pupils  now  receive  training  and  the  study 
of  other  occupations  for  which  the  school  might  offer  training; 
on  the  basis  of  information  under  the  first  two,  the  setting  up 
of  a  curriculum  for  both  the  academic  and  vocational  subjects. 

The  survey  has  been  expanded,  however,  to  include  organi¬ 
zation  and  administration;  the  qualifications  and  duties  of  the 
employees;  the  selection,  housing,  and  salaries  of  teachers; 
nursery  school;  finances;  school  and  home  records;  plant  and 
equipment;  recreation;  extra-curricular  activities;  admission; 
guidance  and  placement  of  children;  follow-up  program  for 
discharged  pupils;  social  problems;  in-service  training  for 
employees,  health  care  and  sanitation;  and  character  and 
religious  education.  The  survey  is  being  directed  jointly  by 
W.  W.  Charters,  director  of  the  Bureau,  and  this  Division.  It 
will  be  completed  during  1932-33. 

The  main  project  in  the  area  of  building  research  was  the 
preparation  of  standards  and  techniques  for  planning  and  evalu¬ 
ating  junior  high-school  plants  in  co-operation  with  William 
E.  Arnold  of  the  Division.  The  data  for  this  project  were 
gathered  by  visits  to  and  inspections  of  thirteen  junior  high 
schools;  interviews  with  principals,  teachers,  and  department 
heads;  and  by  an  examination  and  analysis  of  more  than  five 
hundred  books  and  articles  on  various  phases  of  the  subject. 
This  material  was  then  classified,  analyzed,  and  tentative  stand¬ 
ards  prepared.  These  were  then  sent  to  69  members  of  the 
National  Council  on  Schoolhouse  Construction  and  1 5  promi¬ 
nent  school  architects.  The  tentative  standards  were  then 
revised  in  the  light  of  the  suggestions  and  criticisms  of  these 
two  groups. 

The  first  step  in  constructing  the  score  card  for  evaluation 
was  to  send  to  lOO  junior  high-school  principals,  selected  at 
random  in  eight  states  and  from  schools  of  all  sizes,  blanks  on 
which  they  were  asked  to  give  to  each  of  the  105  items  concern¬ 
ing  the  building  a  numerical  value  by  dividing  a  total  of  1,000 
points  for  the  whole  building.  The  averages  of  these  distribu¬ 
tions  were  taken  as  the  value  assigned  to  each  item  of  the  score 
card.  The  card  was  arranged  so  that  the  rating  of  each  item  can 
be  done  on  the  basis  of  five  steps,  namely,  excellent,  good,  fair, 
poor,  and  very  poor.  The  total  value  assigned  to  the  items  was 
then  divided  among  these  five  steps  on  the  basis  of  a  normal 
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curve  of  distribution.  This  study  will  appear  shortly  as  a 
Bureau  Monograph. 

The  Division  continued  the  usual  practice  of  collecting,  in 
the  fall,  from  Ohio  city  and  exempted-village  school  districts 
information  on  enrollment,  tax  valuation,  tax  levies,  both  for 
schools  and  for  all  purposes,  school  bonded  debt,  bonded  debt 
other  than  school,  special  levies,  and  bond  issues  to  be  submit¬ 
ted  at  the  November  election.  Similar  data  for  the  county 
school  districts  which  were  submitting  special  levies  and  bond 
issues  were  also  collected  and  distributed.  Following  the  elec¬ 
tion  the  results  for  both  the  bond  issues  and  special  levies  were 
secured  for  all  school  districts  in  the  state,  tabulated,  and  dis¬ 
tributed  to  the  districts. 

A  number  of  studies  made  by  others  than  members  of  the 
Division  were  supervised.  Typical  of  these  was  one  undertaken 
by  O.  E.  Fink,  principal  of  the  Hannock  Junior  High  School, 
Zanesville.  This  study  has  to  do  with  the  future  needs  for 
housing  the  junior  and  senior  high-school  pupils  of  the  city  of 
Zanesville.  This  involves,  among  other  things,  an  analysis  of 
expenditures  for  building  at  Zanesville  over  a  number  of  years, 
as  compared  with  a  group  of  other  cities  of  similar  size,  fore¬ 
cast  of  population  and  direction  of  growth,  the  adequacy  of  the 
present  plant,  and  distribution  of  pupils. 


Curriculum  Division 

By  EDGAR  DALE 

The  function  of  the  Curriculum  Division  is  to  co-operate 
with  the  elementary  schools,  high  schools,  and  colleges 
of  the  state  in  the  solution  of  their  curriculum  problems. 
The  assistance  may  be  direct,  through  curriculum  surveys  or 
service,  or  indirect,  through  the  development  of  curriculum 
techniques  which  have  wide  application. 

For  the  past  three  years  the  Bureau  has  been  assisting  the 
State  Department  of  Education  in  the  development  of  a  cur¬ 
riculum  in  health  for  the  secondary  schools  of  the  state.*  Four 
major  ideas  dominated  that  research:  first,  that  the  health 

*  The  £nal  course  of  study  is  available  through  the  State  Department  of  Education 
as  Volume  III  of  the  Health  and  Physical  Education  Series. 
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interests  of  youth  had  not  been  given  sufficient  attention  in  cur¬ 
rent  curriculums.  To  remedy  this  deficiency  approximately 
23,000  questions  dealing  with  health  problems  were  collected 
from  high-school  pupils,  classified,  and  evaluated.  Second,  it 
was  believed  that  finding  and  learning  the  answers  to  these 
problems  was  a  major  responsibility  of  the  students.  Chapter 
references  to  the  best  books  in  the  field  were  therefore  made 
for  many  of  the  problems.  Opportunities  were  offered  whereby 
additional  sources  of  information  not  mentioned  in  the  course 
of  study  might  be  utilized  by  the  pupils.  Third,  it  was  believed 
that  pupils’  interest  alone  was  not  a  completely  valid  standard 
for  the  selection  of  critical  health  problems.  Therefore,  the 
health  problems  which  interested  high-school  boys  and  girls 
were  supplemented  by  those  important  health  problems  in 
which  they  had  shown  slight  or  no  interest.  Fourth,  reading 
activities  were  supplemented  by  a  variety  of  other  methods  of 
gaining  health  experience,  such  as  experiments,  excursions,  and 
discussions. 

Several  studies  were  made  during  the  year  in  vocabulary 
and  reading  research.  Further  experimentation  has  been  made 
of  the  effectiveness  of  the  Dale  Vocabulary  List  as  a  device 
for  the  more  adequate  grade-placement  of  reading  and  radio 
materials.  This  list  of  8,000  common  words  presents  a  famil¬ 
iarity  score  for  each  of  these  words  for  pupils  in  the  fourth, 
sixth,  and  eighth  grades.  Perhaps  the  most  critical  study  dem¬ 
onstrating  the  need  for  such  a  list  was  one  made  of  adult  reading 
with  R.  W.  Tyler.  We  discovered  that  hard  words,  both  tech¬ 
nical  and  nontechnical,  were  by  far  the  best  index  of  the  reading 
difficulty  of  health  materials.  Complex  and  compound  sentences 
had  no  effect  on  the  difficulty  of  a  passage  j  an  increase  in  the 
number  of  simple  sentences  decreased  the  difficulty  of  a  passage; 
and  lengthening  the  sentences  increased  it. 

Why  do  some  parents  give  money  allowances  to  their  chil¬ 
dren.?  Why  do  some  parents  decide  against  it?  Shall  the 
amount  vary  with  age,  sex,  social  environment,  and  the  character 
and  responsibility  of  particular  children?  What  are  desirable 
goals  in  the  financial  training  of  children?  The  answers  to  these 
and  other  related  questions  are  now  being  sought. 

At  the  present  time,  the  contact  of  youth  with  the  field  of 
dramatic  arts  is  through  the  motion  picture.  Indeed,  many 
high-school  students  have  never  seen  a  play  on  the  legitimate 
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cur-  Stage  put  on  by  professional  actors.  Further,  the  present  high- 
itely  school  curriculum  gives  almost  no  assistance  to  pupils  in  the 
:cted  development  of  adequate  standards  of  motion-picture  criticism, 
d,  it  and  the  standards  now  used  by  pupils  to  evaluate  motion  pic- 
hese  I  tures  are  pitifully  meager  and  inadequate.  These  facts,  obtained 
pter  in  a  preliminary  investigation  of  the  motion-picture  tastes  and 
lade  habits  of  high-school  pupils,  led  the  Bureau  of  Educational 
reby  Research  to  develop  a  manual  of  motion-picture  criticism  which 
urse  attempts  to  meet  the  needs  of  those  pupils.  This  manual  was 
;ved  i  experimentally  developed  in  connection  with  English  classes  in 
lard  a  number  of  Ohio  high  schools.  The  aim  in  this  manual  was 
the  not  to  impose  a  set  of  arbitrary  standards  upon  these  boys  and 
^rls  girls.  Instead,  a  series  of  standards  which  they  were  to  use 
s  in  experimentally  as  a  guide  in  developing  their  own  standards  of 
ling  criticism  were  presented  to  them. 

s  of  Techniques  have  been  developed  by  which  the  major  themes 
and  of  motion  pictures  can  be  objectively  discovered,  and  the  per¬ 
centages  of  pictures  of  various  types  have  been  determined  for 
lary  500  pictures  produced  in  1920,  1925,  and  1930.  Further,  a 
lade  more  detailed  analysis  has  been  made  of  the  content  of  a  smaller 
vice  1  number  of  motion  pictures  sampled  from  the  years  1930  and 
idio  1931*  Through  this  analysis  we  have  discovered  the  financial 
mil-  ;  and  occupational  status  of  the  leading  characters,  the  chief  goals 
rth,  which  prompt  their  actions,  the  prevalence  and  nature  of  the 
em-  I  crimes  shown,  the  frequency  and  nature  of  liquor  situations,  the 
ling  '  recreational  life  portrayed  as  attractive,  the  standards  of  living, 
:ch-  I  and  many  other  factors.  The  fundamental  aim  of  the  study,  as 
ling  ■  can  be  seen  from  these  data,  is  to  discover  the  standards  of  life 
aces  shown  in  the  motion  pictures  as  attractive  and  unattractive, 
the 
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Division  of  Accomplishment  Tests 

By  R.  W.  TYLER 

The  primary  interest  of  this  Division  during  the  past 
year  has  been  the  continuation  of  the  work  in  construct¬ 
ing  and  perfecting  means  for  measuring  the  students’ 
attainment  of  those  objectives  of  college  work  which  are  not 
tested  by  the  examinations  and  tests  now  in  use.  Any  adequate 
evaluation  of  the  efiFectiveness  of  instruction  in  a  particular  col- 
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lege  course  or  department  requires  the  use  of  methods  of  j 
measuring  the  results  of  instruction  which  give  valid  evidence  ? 
of  the  degree  to  which  students  are  reaching  each  of  the  impor-  | 
tant  objectives  of  the  course  or  of  the  department.  Usually,  * 
college  instructors  base  their  estimates  of  student  success  upon  | 
subjective  judgments  of  student  ability  as  demonstrated  in  j 
term  papers  and  in  traditional  examinations.  j 

During  the  year  the  Division  has  co-operated  with  twenty 
departments  of  the  University.  The  nature  and  extent  of  assist-  | 
ance  given  vary  widely  with  departments.  The  most  extensive  I 
projects,  probably,  have  been  undertaken  with  the  departments  j 
of  botany,  zoology,  and  home  economics.  These  projects  which  1 
have  been  under  way  for  nearly  three  years  have  involved  a  j 
complete  reorganization  of  the  methods  for  measuring  student  j 
attainment  in  elementary  courses.  The  tests  which  are  being 
developed  in  these  departments  have  been  used  to  diagnose  the 
difficulties  of  entire  classes  in  order  to  determine  the  aspects  of 
the  course  which  should  be  given  greater  emphasis  in  these 
classes;  to  diagnose  the  difficulties  of  individual  students  so  that 
helpful  assistance  might  be  given  students  at  the  points  where  ; 
deficiencies  are  noted;  to  provide  an  analysis  of  student  progress  [ 
which  can  be  understood  by  individual  students  and  which 
serves  to  stimulate  the  students  to  greater  effort  and  to  direct 
their  study  time  efficiently  in  improving  the  phases  of  the  work 
where  progress  is  noticeably  lacking;  to  provide  an  evaluation 
of  the  effectiveness  of  the  instruction  within  the  department  and  i 
with  reference  to  the  work  of  particular  instructors;  to  serve  as 
measuring  instruments  which  can  be  used  in  experimenting  with 
improved  methods  of  teaching  and  of  organizing  the  course  I 
materials;  and  to  give  a  basis  for  student  marks.  j 

In  developing  methods  for  measuring  student  attainment  of  | 
each  of  the  important  objectives  of  college  work  it  is  essential 
that  all  important  objectives  are  included.  Hence,  one  of  the 
fundamental  continuous  projects  of  the  Division  is  to  discover 
the  major  objectives  which  the  colleges  are  attempting  to  reach,  j 
Periodically,  faculty  members  are  consulted;  the  writings  of  | 
college  presidents,  faculty  members,  educational  philosophers,  | 
and  journalists  are  searched;  and  students,  parents,  alumni, 
and  citizens  are  interviewed  to  obtain  statements  of  the  objec¬ 
tives  of  the  colleges.  Those  objectives  which  are  accepted  by  a 
given  college  must  be  considered  in  any  adequate  program  of 
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measurement  of  the  effectiveness  of  the  institution.  Thus  far 
the  various  departments  in  the  University  with  which  we  have 
been  working  have  formulated  ten  major  types  of  objectives, 
each  of  which  will  probably  require  different  types  of  exami¬ 
nations.  These  are: 

Type  A: 

Acquisition  of  information — which  includes  recalling  specific  facts, 
terminology,  and  statements  of  general  principles. 

Type  B: 

Reasoning,  or  utilizing  scientific  method — which  includes  induction, 
testing  hypotheses,  and  deduction. 

Type  C: 

Locating  relevant  data — which  includes  a  knowledge  of  sources  of 
usable  data  and  skill  in  getting  information  from  appropriate  sources. 
Type  D: 

Skills  characteristic  of  particular  subjects — which  includes  laboratory 
skills  in  the  sciences,  mathematical  skills,  and  the  like. 

Type  E: 

Applying  standards  to  technical  performance — which  includes  the 
knowledge  of  appropriate  standards,  ability  to  evaluate  the  relative 
importance  of  several  standards  which  apply,  and  skill  in  applying 
these  standards. 

Type  F : 

Making  reports — which  includes  the  necessary  skill  in  reporting  proj¬ 
ects  in  engineering,  or  reporting  experiments  in  science  and  the  like. 
Type  G: 

Consistency  in  applying  a  point  of  view — which  is  deemed  important 
in  courses  in  philosophy. 

Type  H: 

Interest  in  important  social  problems — which  includes  interest  in 
reading  about  and  in  investigating  problems. 

Type  I: 

Enjoyment  in  wholesome  recreational  activities — which  includes 
enjoyment  of  literature,  music,  and  art. 

TypeX: 

Character — which  is  inclusive  and  may  well  involve  many  specific 
factors. 

Another  research  project  of  some  importance  begun  during 
iV  the  past  year  is  an  attempt  to  determine  satisfactory 
methods  of  selecting  controlled  groups  for  experimentation  with 
college  instruction.  In  most  controlled  experimentation,  two 
groups  of  students  are  chosen,  usually  on  the  basis  of  similar 
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intelligence-test  scores  and  initial  achievement-test  scores.  One  i 
class  is  taught  by  one  procedure  and  the  other  class  by  another.  1 
Any  difference  in  the  final  achievement-test  scores  of  the  two 
classes  is  assumed  to  be  due  to  difference  in  the  specified  pro-  , 
cedure.  This  assumes  that  the  two  groups  as  selected  would 
make  the  same  final  achievement-test  scores  were  they  taught 
by  the  same  procedures.  A  check  of  the  data  from  several  of 
these  studies  revealed  the  fact  that  the  usual  controls  were  not 
adequate  to  guarantee  that  the  two  groups  would  make  the  same 
final  achievement-test  scores  when  taught  by  the  same  pro¬ 
cedures. 

Another  project  of  the  Division  is  concerned  with  develop¬ 
ing  tests  capable  of  giving  finer  units  of  measurement.  In 
using  examinations  to  evaluate  the  effectiveness  of  various 
instructional  procedures,  of  various  teaching  materials,  and  of 
various  administrative  conditions,  it  is  necessary  to  develop 
measuring  instruments  which  will  reveal  small  differences  in 
achievement.  Many  experiments  are  conducted  for  only  a 
quarter  or  a  semester.  In  this  period  of  time  it  is  significant 
to  know  whether  one  group  is  as  much  as  one  week  or  two 
weeks  in  advance  of  the  other.  This  requires  tests  which  will 
give  reliable  evidence  of  such  slight  superiority  in  learning.  At 
present  many  experiments  are  being  conducted  with  measuring 
instruments  so  crude  that  one  group  would  need  to  be  nearly 
six  weeks  in  advance  of  the  other  group  in  order  to  have  reliable 
evidence  of  superiority  from  the  tests.  Without  accurate, 
refined  instruments,  experimentation  is  tremendously  handi¬ 
capped.  Hence,  during  the  past  year  tests  have  been  constructed 
and  used  in  zoology  and  home  economics  which  are  capable  of 
revealing  slight  differences  in  achievement. 

One  interesting  service  project,  begun  in  June,  1932,  is  a 
study  of  the  permanency  of  learning  in  zoology.  Students  who 
had  taken  elementary  zoology  more  than  a  year  before  and  had 
not  taken  any  courses  in  the  subject  since,  were,  without  warn- 
ing,  given  re-examinations  in  the  subject.  Characteristic  differ¬ 
ences  were  found  in  student  retention  among  the  various  types 
of  examination  exercises.  The  students  did  not  show  any  loss 
during  this  period  in  the  ability  to  apply  important  principles 
to  new  situations  and  the  ability  to  interpret  experiments. 

During  the  year  the  Division  has  assembled  a  report  of  the 
attempts  made  by  various  departments  in  the  University  to 
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improve  the  eflFectiveness  of  instruction.  This  volume  has 
recently  been  issued  under  the  title  Service  Studies  in  Higher 
Education.  It  describes  the  reorganization  of  the  elementary 
courses  in  botany,  zoology,  applied  English,  and  textiles,  the 
provisions  made  for  the  superior  students  in  German,  Romance 
languages,  and  zoology,  the  varied  instructional  procedures 
developed  in  history,  botany,  home  economics,  and  zoology,  and 
the  methods  employed  in  the  various  colleges  for  adjusting  the 
college  to  the  Freshmen  and  the  Freshmen  to  the  college.  The 
report  also  illustrates  the  significance  of  careful  measurements 
in  a  program  aimed  to  improve  college  teaching. 

A  project  of  great  potential  significance  to  secondary  and 
higher  education  was  completed  during  the  year.  This  was  a 
study  of  the  effectiveness  of  practice  exercises  when  used  by 
students  to  develop  the  mathematical  abilities  essential  for  the 
field  of  dairy  technology.  The  types  of  mathematical  problems 
and  processes  commonly  encountered  in  this  field  were  deter¬ 
mined.  It  was  found  that  the  mathematics  required  was  not 
more  complex  than  elementary  algebra,  but  nevertheless  stu¬ 
dents  were  usually  unable  to  show  satisfactory  proficiency  in 
these  mathematical  abilities  even  after  taking  courses  in  college 
algebra.  By  the  use  of  a  series  of  exercises  which  gave  students 
practice  in  solving  the  various  types  of  problems  it  was  possible 
to  attain  the  desired  proficiency  without  setting  up  an  additional 
course;  thereby  saving  instructional  expense  and  student  time. 
There  may  be  other  elementary  abilities  demanded  in  college 
or  high-school  work  which  can  be  economically  and  effectively 
handled  in  the  same  way. 

Personnel  Division 

By  W.  H.  COWLEY 

The  Personnel  Division  of  the  Bureau  of  Educational 
Research  is  organized  not  only  to  conduct  investigations 
in  student-personnel  administration,  but  also  in  general 
administrative  research.  Problems  in  both  of  these  fields  are 
continuously  being  studied,  the  majority  of  them  relating 
to  the  internal  organization  of  Ohio  State  University.  As 
illustrative  of  personnel  research  are  the  investigations  in 
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admissions,  the  eflFect  upon  freshman  scholarship  of  mid-quarter 
interviews  in  the  College  of  Engineering,  the  study  of  student 
mores  in  a  half-dozen  colleges  and  universities  of  the  country, 
the  preparation  of  a  bulletin  for  the  high-school  students  of  the 
state  entitled  “Preparing  for  College,”  the  compilation  of  The 
Personnel  Bihlto graphical  Index,  the  survey  of  the  personnel 
activities  of  Ohio  State  University  with  a  view  to  their  co¬ 
ordination,  and  the  development  of  rating  scales  for  various 
purposes. 

Any  one  of  these  investigations  would  require  many  para¬ 
graphs  of  description.  A  brief  summary  of  The  Personnel 
Bibliographical  Index  is,  perhaps,  sufficient  to  illustrate  the 
ramifications  of  these  studies.  The  Index  is  a  volume  of  433 
pages,  which  was  published  by  the  Bureau  in  the  spring  of  1932. 
During  the  past  fifteen  years,  literally  thousands  of  books  and 
articles  have  been  written  upon  student-personnel  problems. 
The  literature  is  fast  becoming  so  numerous  that  the  average 
administrator  and  research  worker  finds  it  impossible  to  keep 
in  touch  with  all  the  research  and  administrative  developments 
in  this  field.  In  1930,  therefore,  the  Personnel  Division  under¬ 
took  the  reading  and  annotation  of  this  literature.  More  than 
two  thousand  of  the  better  books  and  articles  have  been  anno¬ 
tated,  indexed,  and  cross-referenced  in  The  Personnel  Biblio¬ 
graphical  Index.  Already  the  volume  has  had  a  large  sale,  not 
only  among  college  administrators,  but  also  among  investigators 
and  administrators  in  secondary  schools.  The  Division  plans  to 
publish  biannual  supplements  to  keep  this  Index  up  to  date  as  a 
reference  tool  for  interested  workers. 

The  administrative  research  conducted  by  this  Division  may 
be  illustrated  by  citing  three  studies:  calendar  organization 
of  Ohio  State  University,  the  effectiveness  of  the  Ohio  State 
Educational  Conference,  and  the  development  of  administrative 
offices  in  institutions  of  higher  education.  The  calendar  study 
was  completed  by  this  Division  just  a  year  ago,  and  this  fall 
it  is  being  published  by  the  University. 

The  study  of  the  Ohio  State  Educational  Conference  fol¬ 
lowed  as  a  natural  result  of  the  administration  of  the  Confer¬ 
ence  by  this  Division  of  the  Bureau.  For  twelve  years  Ohio 
State  University  has  been  playing  host  each  spring  to  the  edu¬ 
cators  of  the  state  of  Ohio.  There  has  been  no  question  in  the 
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I  minds  of  anyone  of  the  desirability  of  this  Conference,  but  this 

1  I  last  year  an  evaluation  was  made  in  order  to  bring  together  on 
paper  the  judgments  of  2,000  of  the  6,500  people  who  attended 
the  Conference  in  1932.  The  results  of  the  investigation  have 
demonstrated  that  the  Ohio  State  Educational  Conference  is 
annually  making  a  contribution  to  education  in  the  state  of 
Ohio,  the  importance  of  which  can  hardly  be  overestimated. 

The  study  of  the  rise  and  development  of  administrative 
offices  in  institutions  of  higher  education  is  a  natural  develop¬ 
ment  of  the  interest  of  the  Division  in  the  history  of  student- 
personnel  offices  in  colleges  and  universities.  It  is  often  thought 
that  the  student-personnel  movement  is  new,  but  although  new 
I  techniques  and  terminologies  have  been  developed  within  the 
past  two  decades,  the  concept  is  as  old  as  our  oldest  institution. 
In  order  that  educators  over  the  country  may  recognize  the 
integral  relationship  between  personnel  administration  and  the 
educative  process,  this  historical  investigation  has  been  under¬ 
taken,  and  it  has  now  developed  to  include  all  administrative 
offices  since  all  are  so  closely  related  in  organizations. 


Radio  Division 

I  F.  H.  LUMLEY 

The  Radio  Division  of  the  Bureau  of  Educational  Re¬ 
search  is  carrying  on  its  research  work  under  a  grant 
from  the  Payne  Fund  of  New  York  City.  Investiga¬ 
tions  of  two  kinds  are  being  conducted;  the  collection,  evalua¬ 
tion,  and  testing  of  methods  of  measuring  the  audience’s 
response  to  radio  presentations;  and  the  determination  of  the 
effect  of  specific  radio  presentations,  such  as  the  WE  AO  French 
lessons  and  the  Ohio  &hool  of  the  Air  geography  talks. 

At  the  present  time  printed  copies  or  statements  of  surveys 
made  by  the  methods  of  mail  response,  questionnaire,  telephone 
interview,  and  personal  interview  have  been  obtained.  An 
analysis  is  being  made  of  these  methods  to  determine  the  most 
probable  sources  of  error,  the  actual  errors  found,  the  economy 
of  operation,  and  suggestions  for  improvement.  When  all  of 
this  information  is  assembled  completely,  it  will  be  presented 
in  monograph  form. 
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A  survey  has  been  made  of  the  WEAO  radio  audience  to 
determine  the  time  at  which  this  audience  prefers  to  listen  to  ed¬ 
ucational  programs,  the  times  at  which  they  actually  listen  to 
programs,  preference  for  different  types  of  radio  programs,  and 
the  reliability  of  audience  reports.  This  survey  has  been  issued 
in  mimeographed  form.  The  questionnaire  method  was  used 
in  making  the  survey.  Further  information  on  the  listening 
habits  of  the  radio  audience  is  being  gathered  by  means  of  a 
telephone  survey  in  co-operation  with  a  research  agency. 

The  co-operation  of  five  teachers  of  French  has  been  se¬ 
cured  in  connection  with  the  WEAO  radio  broadcasts.  Records 
were  made  of  the  pupils’  French  pronunciation  before  hearing 
the  broadcasts  and  afterward.  Control  hearing  groups  under  the 
same  teacher  are  used  to  check  the  results.  Furthermore,  the 
pupils  have  made  reports  on  the  broadcast  lessons,  and  many 
suggestions  for  improvement  have  been  obtained. 

During  the  year,  approximately  160  teachers  and  their 
pupils  received  either  one  or  all  of  the  series  of  tests  given  in 
connection  with  Mr.  McConnell’s  geography  talks.  The  pur¬ 
pose  of  these  tests  was  to  ascertain  whether  or  not  relationships 
could  be  established  between  certain  factors,  such  as  types  of 
emphasis  used,  spelling  of  words,  repetition  of  materials,  giving 
of  numbers  and  dates,  use  of  maps,  and  the  like.  The  tests  were 
of  the  compound  multiple-choice  (or  master  list)  variety,  and, 
with  the  exception  of  one  special  test,  were  all  on  information 
or  logical  deduction.  From  the  results  of  this  test  material,  it 
is  hoped  that  specifications  may  be  set  up  with  reference  to  types 
of  materials  which  can  be  given  over  the  radio.  In  order  to 
check  conclusions  from  the  tests  on  the  radio  talks,  copies  of  the 
talks  were  printed.  A  group  of  control  children  read  these  talks 
and  took  the  same  tests. 

The  Radio  Division  has  been  interested  in  several  other 
projects.  In  co-operation  with  the  Ohio  School  of  the  Air  it  has 
devised  or  helped  to  devise  report  forms  to  be  filled  out  by 
teachers  using  radio  lessons  of  the  School  of  the  Air.  In  co¬ 
operation  with  the  Department  of  Romance  Languages,  at  Ohio 
State  University,  it  has  been  interested  in  determining  the  best 
ways  of  furnishing  the  teacher  with  information  about  the 
WEAO  foreign-language  lessons.  The  Radio  Division  has  also 
taken  active  part  in  arranging  meetings  for  the  annual  Institute 
for  Education  by  Radio  held  at  Ohio  State  University. 


IT  IS  inevitable  that  information  concerning  the  work  of 
the  Bureau  of  Educational  Research  should  be  desired 
by  interested  persons.  The  Educational  Research 
Bulletin  is  one  means  by  which  the  Bureau  makes 
known  the  problems  on  which  it  is  working  and  the  results 
which  are  being  obtained.  To  present  the  activities  of  the 
Bureau  more  informally,  staff  members  will  give  a  series 
of  radio  talks  over  the  Ohio  State  University  Radio  Sta¬ 
tion,  WEAO.  In  each  talk,  some  special  phase  of  educa¬ 
tional  research  will  be  considered  and  the  work  of  the 
Bureau  related  to  the  need  for  such  research.  During 
October  the  following  talks  will  be  given: 

Friday,  October  14. 

“Do  Children  Spell  as  Well  as  They  Used  to?  ” 

W.  W.  Charters. 

Friday,  October  21. 

“Is  Teaching  as  Good  as  It  Used  to  Be?” 

W.  W.  Charters. 

Friday,  October  28. 

“What  Will  Be  the  Demand  for  Teachers  in  the  Future?” 
Earl  W.  Anderson 

All  talks  are  given  on  Fridays  at  7:45  p.m.  over  WEAO 
(570  kilocycles). 


